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ABSTRACT 


Teaching competence is an important skill for graduate students to acquire 
and is often considered a precursor to an academic career. In this study, we 
evaluated the effects of a multi-day teaching workshop on graduate teach¬ 
ing philosophies by surveying 200 graduate students, 79 of whom had taken 
the workshops and 121 who had not. We found no difference between groups 
(workshop attendees versus non-attendees) in their beliefs that (a) it is im¬ 
portant to focus on in-depth learning of core concepts when teaching and (b) 
“memorization” is a poor learning strategy for students. On average, however, 
respondents who had taken the workshop allocated more in-class time for 
student-to-student discussions (interactive engagement) and placed less em- 
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phasis on lecturing. These results suggest that graduate students are gener¬ 
ally aware of the importance of conceptual learning, but workshop attendees 
have clearer ideas on howto teach for effective learning. 

RESUME 

La capacite d’enseigner est une competence importante pour les etudiants en 
formation doctorale et est souvent consideree comme un attribut necessaire a 
la poursuite dune carriere academique. Lors de cette etude, nous avons evalue 
les effets d’un atelier de developpement pedagogique de plusieurs jours sur la 
philosophic d’enseignement des etudiants en these en interrogeant 200 sujets 
- 79 qui avaient assiste aux ateliers et 121 qui n’y avaient pas participe. Nous 
n’avons trouve aucune difference entre les groupes (ceux qui ont participe 
a l’atelier par rapport aux non-participants) dans leur croyance que (a) lors 
de l’enseignement, il est important de se concentrer sur l’apprentissage 
en profondeur des concepts principaux et (b) la «memorisation» est une 
mauvaise strategie d’apprentissage pour les etudiants. Cependant, en 
moyenne, les repondants qui avaient participe a l’atelier ont consacre plus de 
temps a des discussions entre etudiants (engagement interactif) et ont accorde 
moins d’importance aux cours magistraux. Ces resultats suggerent que les 
etudiants de niveau doctoral sont generalement conscients de l’importance 
de l’apprentissage conceptuel, mais que ceux qui ont participe aux ateliers ont 
des idees plus claires pour faciliter un tel apprentissage. 

Over the last decade, North American universities and colleges have taken a new di¬ 
rection in graduate education by implementing professional development programs for 
their graduate students. In 2008, the Canadian Association for Graduate Studies (CAGS), 
the Society for Teaching and Learning in Higher Education, and the Canadian federal 
government’s Tri-Council funding agencies supported this initiative by starting a discus¬ 
sion of priorities, necessary skills, and actions to move the dialogue forward. This group 
promotes the development of programs for graduate students to ensure they acquire by 
the end of their degree the skills to perform well in what has become known as the knowl¬ 
edge economy (CAGS, 2008). In a 2010 benchmarking activity (Jenkins, 2010), more 
than 30 institutions were surveyed to assess the support for developing such skills. It was 
found that each of the schools surveyed had initiatives that ran parallel to the academic 
program and aimed at improving the graduate student skills identified by CAGS. Many of 
the institutions offered activities to help graduate students build the teaching skills enu¬ 
merated as the characteristics of a Highly Qualified Person (CAGS, 2008). 

Teaching competence is increasingly recognized in academics as a complement to re¬ 
search skills (Brew, 2003; Prince, Felder, & Brent, 2007). The definition of teaching com¬ 
petence can vary from one discipline to another, but it broadly involves the facilitation 
of collaborative and cooperative learning among students (McKeachie, 2007). For newly 
graduated PhDs, an expectation of teaching competence generally emanates from teach¬ 
ing experiences completed during their degree. Graduate students become involved as 
teaching assistants, assistant/guest lecturers, lab instructors, teaching fellows, and even 
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full course instructors. As graduate programs are often a precursor to academic careers 
where a high level of teaching competency is expected, graduate students who develop 
these skills while in their graduate program will be better prepared for a professorship 
when they graduate. 

The characteristics of effective in-class teaching have been well established and go far 
beyond good presentation skills (Armbruster, Patel, Johnson, & Weiss, 2009; Crouch & 
Mazur, 2001; Smith et al., 2009). Graduate students typically develop oration skills for 
presentations at conferences, but these skills are different from good teaching (Mazur, 
2009; McKeachie, 2007). Among other features, good teaching promotes peer-learning 
opportunities (Crouch & Mazur, 2001) and in-class student-to-student (peer) interac¬ 
tion (McKeachie & Svinicki, 2010). Teaching that incorporates these features has a better 
chance of obtaining the desired learning outcomes (Davies, 2002; Ernst & Colthorpe, 
2007), and research shows that peer interaction stimulates high-level cognitive activity 
leading to higher learning gains (Cohen, 1994; O’Donnell & King, 1999; Webb & Palincsar, 
1996). In a meta-analysis comparing interactive engagement pedagogies to traditional 
lecture-only pedagogies, Hake (1998) showed that the former consistently outperformed 
the latter as reflected in students’ pre- to post-course knowledge evaluations. 

There are now three popular approaches pursued, often concurrently, in the devel¬ 
opment of graduate teaching skills at Canadian universities. These approaches are (a) 
independent presentations and short workshops, (b) credit graduate courses, and (c) a 
certificate program where participants engage in a substantial number of the aforemen¬ 
tioned activities in addition to evaluated activities such as reflective writing, literature 
reviews, and teaching. Examples of these programs can be found at Queen’s University 
(Queen’s CTL, 2010), the University of British Columbia (UBC, 2010), and the University 
of Western Ontario (UWO, 2010). It appears that the greatest number of graduate stu¬ 
dents participate in the first of these popular approaches (independent presentations and 
short workshops). Although this approach is well intentioned, it is unclear how effective 
short interventions are in helping graduate students develop effective teaching pedago¬ 
gies. Moreover, similar short workshop formats have been found to be not necessarily 
effective for professors (McAlpine & Winer, 2002). A fourth and less common approach 
is that of multi-day intensive workshops. This approach, demonstrated to be effective for 
professors (Saroyan & Amundsen, 2004), is taken by the Faculty of Science at McGill Uni¬ 
versity (Harris & McEwen, 2009; T-PULSE, 2010), the Faculty of Agriculture at McGill 
(www.mcgill.ca/miti), and by the Department of Physics and Astronomy at UBC (http:// 
www.physics.ubc.ca/~phas_ta/). 

The objective of this paper is to test the effectiveness of multi-day pedagogical devel¬ 
opment workshops on graduate teaching practice and philosophies at a research-inten¬ 
sive university in eastern Canada. The aim of these workshops is to provide participants 
with teaching strategies they can immediately apply to their teaching situation, and to 
facilitate participants’ development of their own teaching philosophy by engaging them 
in activities focused on a constructivist, learning-centred approach. These elements, com¬ 
bined with time for reflection on personal approaches toward teaching, are designed to 
help with the development of a teaching philosophy. 


CJHE / RCES Volume 42, No. 1, 2012 


Putting Research Into Practice / P. J. T. White, D. Syncox, A. Heppleston, S. Isaac & B. Alters 101 


METHODS 

At McGill University, pedagogy development workshops for science graduate students 
were started in 2003 by the Tomlinson Project in University-Level Science Education 
(T-PULSE, 2010). Over the past nine years, the workshops have evolved to include mod¬ 
ules on learning outcomes, interactive engagement strategies, learner diversity, formative 
evaluation, and grading. Each multi-day workshop (in its entirety) incorporates approxi¬ 
mately 12 hours of training. The curricula for the workshop modules were researched and 
created by graduate student teaching fellows working for T-PULSE, who at times collabo¬ 
rated with faculty members from McGill’s Teaching and Learning Services and Faculty 
of Education. Currently, the workshops are facilitated by a group of graduate student 
teaching fellows from scientific disciplines under the supervision and guidance of interim 
director Dr. Nicholas de Takacsy and the T-PULSE office. (The former director, Dr. Brian 
Alters, is a co-author on this paper.) From 2003 to 2011, the workshops have been offered 
more than 30 times and have engaged more than 750 graduate student participants. In 
that time, more than 25 different graduate students have served as teaching fellows on the 
workshop facilitation team. 

We collected two datasets for this study. The first dataset was collected from a group 
of 40 graduate students at the January 2009 edition of the T-PULSE pedagogy develop¬ 
ment workshops. We conducted pre- and (immediate) post-workshop surveys to gauge 
how the teaching philosophies of participants may have changed over the course of the 
two-day event. These responses will be referred to as pre-workshop and post-workshop 
responses throughout the rest of this paper. The second dataset was collected in February 
2009. We sent an electronic online survey to approximately 850 graduate students across 
the departments of the Faculty of Science at McGill University. This pool of students com¬ 
prised master’s and PhD candidates who had or who had not taken the T-PULSE peda¬ 
gogy development workshops. Participation in the survey was anonymous or confiden¬ 
tial (depending on whether respondents wanted to be entered into a prize-draw). Data 
from these online surveys will be referred to as YWS-online and NWS-online responses 
throughout the rest of this paper. YWS stands for Yes Workshop and includes answers 
from online respondents who had taken the multi-day T-PULSE workshops. NWS stands 
for No Workshop and includes answers from online respondents who had not taken the 
multi-day T-PULSE workshop or those who had taken minimal pedagogical training 
through independent presentations or short workshops; that is, they had participated 
in the first popular approach identified earlier. The survey we used in both sets gathered 
data on respondent conceptions of good teaching and the value of lecturing as an effec¬ 
tive teaching and learning strategy. By using the same survey instrument for our Janu¬ 
ary 2009 and February 2009 surveys, we sought to determine if the workshops actually 
caused a change in teaching philosophy, or if those who attended the workshops were 
simply predisposed to having interactive teaching philosophies already in place. 

In our survey, we asked three demographic questions and six informative teaching- 
philosophy questions. The demographic questions gathered data on the respondents’ 
teaching development activities and teaching experience, including a self-assessment of 
teaching abilities. The teaching-philosophy questions gathered data on what the respon¬ 
dents identified as good teaching by asking them to list the qualities of good teaching 
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and to identify the primary responsibilities of instructors. These latter questions were 
designed to allow us to identify objectivist versus constructivist teaching philosophies 
among respondents. See Appendix 1 for the full survey instrument. 

We used t tests and MANOVA tests to analyze the survey response data. T tests are ap¬ 
propriate statistical tools for testing whether the means of two sets of responses are differ¬ 
ent. We used this test for Questions 2 and 3 (separately) to compare the numerical spread 
of responses among workshop participants versus non-workshop participants (for exam¬ 
ple, a = 5, b = 4, c = 3, d = 2, e = 1). AMANOVA test (Multiple Analysis of Variance) is ap¬ 
propriate for testing whether the means of multiple dependent variables vary with respect 
to a categorical independent variable. In our case, this tool allows us to test whether the 
answer profiles from Questions 6, 7, 8, and 9 (ensemble) differ among workshop partici¬ 
pants versus non-workshop participants. The advantage of using this type of analysis is 
that it tests for differences in many questions simultaneously, thus reducing the number 
of statistical tests required. When a significant p value is returned, the means of the four 
dependent variables are analyzed to determine which variables are the likely causes of the 
significant difference. For these tests, we used a similar scoring system (that is, a = 5, b 
= 4, and so on), as in Questions 2 and 3. Question 9, however, was reverse-scored so that 
higher values corresponded to more time allotted for peer-to-peer interactions (that is, a 
= 1, b = 2, c = 3, d = 4, e = 5). Although the categories in Question 9 are not equal interval 
categories, we opted to use this scoring so that it would match the scoring used in Ques¬ 
tions 6 to 8. Furthermore, numerical categories matching the average minute-values of 
each category were not optimal because the fifth category had an unspecified upper limit 
(that is, more than 20 minutes ). 

To further explore our data, we performed three additional MANOVA tests relating to 
the answer profiles for Questions 6, 7, 8, and 9. First, we tested for differences between 
pre-workshop responses and NWS-online responses; that is, neither group had taken the 
T-PULSE workshops. Second, we tested for differences between post-workshop responses 
and YWS-online responses; that is, both groups had taken the workshops. Third, among 
YWS-online responses, we tested to see if answer profiles for Questions 6, 7, 8, and 9 dif¬ 
fered based on the amount of time that had passed since the workshop was taken; that is, 
we based our test on the data collected in Question i(a)(i). To facilitate this analysis, we sub¬ 
divided YWS-online responses into four groups corresponding to those who took the work¬ 
shop in the current academic year, one year ago, two years ago, or more than two years ago. 

For all of the aforementioned statistical analyses (the two t tests and the 5 MANOVA 
tests), we used Bonferroni-adjusted alpha levels of .007 (.05/7 total tests) to determine 
statistical significance. Statistical analyses were not run on Questions 4 and 5. Instead, 
the results are displayed in table and graph format (respectively) for visual comparison. 

RESULTS 

Of the 850 graduate students we targeted with our February 2009 online survey, 200 
responded. Of these students, 79 were former T-PULSE workshop participants (YWS- 
online) and 121 were not (NWS-online). Of the 79 YWS-online respondents, 27 had taken 
the workshop in January 2009,14 had taken it one year prior, 22 had taken it two years 
prior, and 16 had taken it more than two years prior. Of the 121 NWS-online respondents, 
93 had never participated in formal pedagogy development training (workshops, semi- 
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nars, and so on). For the sake of simplicity, we will refer to all 121 respondents in this 
category as NWS-online respondents. YWS-online respondents had 2.6 years of teach¬ 
ing experience on average compared to 2.0 years of experience on average among NWS- 
online respondents. This difference was not significant (Question 2: t = 2.26, SEM = 
.18, p = .026). YWS-online respondents rated their teaching ability as 4.8/7 on average 
compared to 4.4/7 on average among NWS-online respondents. This difference was also 
not significant (Question 3: t = 0.88, SEM = .15, p = .38). 

In our February 2009 online survey, when we asked respondents to list qualities of 
good teaching (Question 4), YWS-online respondents more frequently listed Interactive 
Engagement and Results in Student Learning, whereas NWS-online respondents more 
frequently listed Good Rapport With Students and Innovative/Energetic (Table 1). When 
asked what the Primary Responsibilities of a course instructor should be (Question 5), 
YWS-online respondents chose Encourage Peer-Peer Interaction more often than NWS- 
online respondents (Figure 1). YWS-online respondents chose Deliver a Good Lecture 
and Make Sure Course Material Is Current less frequently than did NWS-online respon¬ 
dents, but these differences were less pronounced. In Figure 1, the dashed lines separate 
the online survey data (left) from the data collected at the January 2009 workshop (right). 
The large difference in percentages between the two datasets (for example, in the Deliver 
a Good Lecture category) is present because online respondents were asked to “choose all 
that apply” and pre-post workshop respondents were asked to choose only one answer. 
The online data that are presented therefore show the percent of responses, whereas the 
pre-post workshop data show the percent of respondents. 

Table 1. 

Qualities of Good Teaching Identified by Survey Respondents 


Qualities of Good Teaching 

NWS* Responses (%) 

YWS* Responses (%) 

Good lecturing 

42 

38 

Uses interactive engagement 

11 

28 

Good knowledge of subject matter 

10 

9 

Good rapport with students/asks and answers questions 

32 

23 

Patience 

15 

6 

Tracks student progress during course 

8 

6 

Encourages learning independence 

7 

6 

Innovative/energetic/enthusiastic 

26 

21 

Awareness of diverse learner backgrounds 

5 

10 

Uses good evaluation methods 

6 

6 

Well organized/prepared 

11 

9 

“Clarity” 

5 

6 

Results in student learning 

2 

12 

Other 

1.3 

15 


Note. *NWS responses were from respondents who had not taken the T-PULSE teaching workshops; YWS 
responses were from respondents who had taken the T-PULSE teaching workshops. 
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Figure 1. Primary Responsibilities of a Course Instructor as Listed by Survey Respondents 


In the pre-post dataset, MANOVA analyses indicated that the way pre-workshop re¬ 
spondents and post-workshop respondents answered Questions 6, 7, 8, and 9 (ensemble) 
was significantly different (F 4 yo = 4.9, MSE = 4.7, p = .0016). The biggest differences 
between these two respondent groups were in their answers to Questions 6 and 9. Post¬ 
workshop respondents agreed less with the effectiveness of lecturing (average response 
3.0/5 post-workshop versus 2.4/5 pre-workshop) (Table 2) and allotted more in-class 
time for peer-to-peer interactions (average response 3.1/5 post-workshop versus 2.2/5 
pre-workshop) (Figure 2). Similarly, in the online dataset, a MANOVA analysis found 
that the answers given by YWS-online respondents and the answers given by NWS-online 
respondents also differed significantly (F 4 = 7.0, MSE = 4.4, p < .0001). YWS-online re¬ 

spondents agreed less with the effectiveness of lecturing (average response 2.6/5 for YWS 
versus 3.1/5 for NWS) (Table 2) and allotted more in-class time for peer-to-peer interac¬ 
tions (average response 2.9/5 for YWS versus 2.2/5 for NWS) (Figure 2). In Figure 2, the 
dashed lines separate the online survey data (left) from the data collected at the January 
2009 workshop (right). 

MANOVA analyses did not, however, identify significant differences in the way that 
pre-workshop respondents and NWS-online respondents answered Questions 6,7, 8, and 
9 (F 4i53 = 0.49, MSE = 4.5 p = .74). Neither was there a significant difference in the way 
that post-workshop respondents and YWS-online respondents answered this set of ques¬ 
tions (F = 1.9, MSE = 4.2 p = .12). Among YWS-online responses, the time since the 
workshop was taken was not significantly related to differences in answer profiles (F _ = 
1.8, MSE = 4.3, p = .13). 
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Table 2. 

Teaching and Learning Beliefs Among Survey Respondents 


6 . In general, listening to lectures is one of the best ways for students to understand course material. 

5. Strongly 4. Agree 3. Neutral 2. Disagree 1. Strongly Average 
agree (%)(%)(%)(%)disagree ( 96 ) score 


Pre-workshop 

10 

30 

20 

28 

13 

3-0 

Post-workshop 

5 

10 

15 

55 

15 

2.4 

NWS-online 

11 

27 

28 

28 

6 

3-1 

YWS-online 

8 

18 

22 

38 

15 

2.6 


7. It is better to cover a lot of course content than to promote a deep understanding of a few key con- 
cepts. 



5. Strongly 
agree ( 96 ) 

4. Agree 
( 96 ) 

3. Neutral 
( 96 ) 

2 . Disagree 
( 96 ) 

1. Strongly 
disagree ( 96 ) 

Average 

score 

Pre-workshop 

0 

13 

23 

49 

15 

2.3 

Post-workshop 

3 

15 

10 

51 

21 

2.3 

NWS-online 

1 

8 

18 

50 

23 

2.1 

YWS-online 

0 

9 

15 

53 

23 

2.1 


8. In introductory courses, instructors should encourage “memorization” as one effective strategy for 
learning core concepts. 



5. Strongly 
agree ( 96 ) 

4. Agree 
( 96 ) 

3. Neutral 
( 96 ) 

2. Disagree 
( 96 ) 

1. Strongly 
disagree ( 96 ) 

Average 

score 

Pre-workshop 

5 

10 

8 

46 

3 i 

2.0 

Post-workshop 

0 

10 

8 

5 i 

3 i 

2.2 

NWS-online 

4 

16 

21 

32 

27 

2.4 

YWS-online 

4 

13 

25 

35 

23 

2.4 


60 




□ Online Respondent, 
No Workshop 
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with Workshop 

H Pre-Workshop 
Survey 

□ Post-Workshop 
Survey 



Omins 1-5 mins 6-10 mins 11-20 mins more than 20 
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How much time would you set aside for peer-peer interactions in a 60 minute class? 


Figure 2. The Amount of Time Allotted by Survey Respondents for Peer-Peer Interactions 
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DISCUSSION 

The different groups of graduate students surveyed had convergent ideas on what con¬ 
stitutes good learning—particularly that memorization is an insufficient learning technique 
for acquiring deep understanding of key concepts. There was, however, an important dif¬ 
ference in perspective between those who did and those who did not attend the multi-day 
T-PULSE workshops on the utility of lectures. Those who did not attend the workshop held 
the position that lectures are an effective way to learn. This view expresses a teacher-cen¬ 
tred approach without significant focus on student learning or peer-peer interactions. In 
contrast, those who took the workshop expressed a more learning-centred approach in that 
they found lecturing to be of limited value for student learning. These respondents were 
also likely to plan more time for in-class peer discussion (Figure 2). These two components 
(more interactive engagement and less focus on lecturing) have been well established as 
part of a high-value teaching philosophy that leads to higher learning gains (Brooks, 1984; 
Hake, 1998; Mazur, 2009; Smith, et al., 2009). Thus, it seems that graduate students who 
have taken the workshops not only have a good idea of what constitutes good teaching but 
also howto accomplish it in practice. This finding suggests that a successful workshop mod¬ 
el should educate attendees about effective learning-centred pedagogies rather than teach¬ 
ing attendees the merits of deep conceptual learning as compared to those of memorization. 

The general consistency of the answers of post-workshop respondents (January 2009) 
and YWS-online respondents demonstrates that a pedagogy change among attendees is 
not limited to a temporary change occurring immediately post-workshop. There was also 
no statistical evidence that the change in teaching philosophy erodes with time. The ex¬ 
ception to this was that post-workshop respondents disagreed slightly more with the util¬ 
ity of memorization than did online respondents who had taken the workshop (Table 2). 
Both sets of responses were still negatively disposed toward memorization. There also 
seemed to be a small difference in the amount of time that post-workshop respondents 
would give for in-class discussion compared to online respondents who had taken the 
workshop (Figure 2). This difference did not result in significant MANOVA analyses and 
only represents a marginal difference, as the most common tendency in both groups was 
to allow six to 10 minutes of interaction in each 60-minute class. These very small differ¬ 
ences notwithstanding, the overall suite of results suggests that the T-PULSE workshops 
at McGill University assist graduate students to develop key components of their teaching 
philosophy, better preparing them for their current teaching roles and future professorial 
careers. Research suggests that professors who learn about effective teaching strategies 
early in their careers are more inclined to permanently adopt a learning-centred approach 
(Gandell, Weston, Finklestein, & Winer, 2000; Weston & Cranton, 1986). 

Although changes in teaching philosophy among graduate students can be facilitated 
through teaching workshops, it can be comparatively difficult to stimulate pedagogical 
change across a department or institution. Our results suggest that a viable solution to 
this problem may require a long-term focus, training the next generation of professors 
to lead the shift to effective in-class pedagogy. The impact of an investment in the peda¬ 
gogical development of graduate students may be enhanced by the broad use of teaching 
portfolios in recruitment and tenure decisions, making well-prepared graduate students 
more competitive in the professorial job market. The importance of training new profes- 
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sors for their teaching roles has been clearly established (Sorcinelli, 1994), particularly 
because new teachers quickly form lasting approaches and attitudes toward teaching 
(Boice, 1996). Because most future professors start teaching in graduate school, targeting 
graduate students with pedagogy development workshops can have a large impact on the 
future quality of instruction at Canadian universities.* 
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APPENDIX I: SURVEY INSTRUMENT 

1. Have you taken a teaching-development workshop or seminar before? 

a. Yes - The T-PULSE Graduate Teaching (Pedagogy Development) 
Workshop 

i. If yes, when did you take the T-PULSE Workshop? 

b. Yes - A McGill Teaching and Learning Services Seminar 

c. Yes - A different McGill teaching seminar or workshop 

d. Yes - a non-McGill teaching seminar or workshop 

e. No 

2. How long have you been teaching for (including time as a teaching assistant)? 

a. I have not taught before 

b. Less than l year 

c. l or 2 years 

d. 3 or 4 years 

e. More than 4 years 

3 . How would you rate yourself as a teacher compared to others at McGill in your 
field? 

a. I am one of the best 

b. I am considerably above average 

c. I am slightly above average 

d. I am average 

e. I am slightly below average 

f. I am considerably below average 

g. I am one of the worst 

h. N/AI have never taught before 

4 . Give two characteristics of good teaching (open ended, asked in the February 
2009 survey only). 

5 . What should the primary responsibilities of a course instructor be (choose two 
options): 

a. To deliver clear and concise lectures 

b. To answer student questions (both in and out of class) 

c. To encourage peer-peer interactions among students 

d. To stay up-to-date on subject matter in their field of instruction 

e. To design appropriate means for evaluating students (quizzes, tests, 
exams) 

f. To design appropriate means for students to give feedback on the 
instructor’s teaching 

6. In general, listening to lectures is one of the best ways for students to understand 
course material (choose one). 
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a. Strongly Agree 

b. Agree 

c. Neutral 

d. Disagree 

e. Strongly Disagree 

7 . It is better to cover a lot of course content than to promote a deep understanding 
of a few key concepts. 

a. Strongly Agree 

b. Agree 

c. Neutral 

d. Disagree 

e. Strongly Disagree 

8. In introductory courses, instructors should encourage “memorization” as one 
effective strategy for learning core concepts.* 

a. Strongly Agree 

b. Agree 

c. Neutral 

d. Disagree 

e. Strongly Disagree 

9 . When planning a 6o-minute lecture, you should typically set aside the following 
amount of time for students to interact with one another about the subject matter 
while they are in class (choose one). ** 

a. o minutes 

b. 1 to 5 minutes 

c. 6 to 10 minutes 

d. 11 to 20 minutes 

e. More than 20 minutes 

* This question was asked differently in the January 2009 workshops: “The 
fundamentals of undergraduate science courses should be taught as a series of facts 
that the students must learn.” 

** We created the response in an attempt to create meaningful and distinct categories. 
We felt this was preferable to an equal-interval approach. 
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